Bạn đang xem: Memory usage
Sometimes the load spikes lớn around 2.0:
However, CPU usage doesn’t actually increase during that period, which also rules out iowait being the cause:
It seems periodic when it happens (eg. In this graph it happened roughly every 20-25 minutes). I suspected a cronjob, but I don’t have any cronjobs that run every 20 mins. I have also tried disabling my cronjobs và the load spike still occurs.
I managed lớn actually see this happening while SSH’d into the server… It had a load of 1.88, but the CPU was 94% idle và there was 0% iowait (which is what I expected the cause might have been)
There does not appear lớn be a lot of disk I/O when this happens.
I"m stumped. Any ideas?
share
Improve this question
Follow
asked Jan 19, 2019 at 18:58
Daniel Lo Nigro
Daniel Lo Nigro
46411 gold badge66 silver badges1010 bronze badges
2
add a phản hồi |
1 Answer 1
Sorted by: Reset to mặc định
Highest score (default) Date modified (newest first) Date created (oldest first)
9
So I worked this out... It turns out it was caused by the software I was using to monitor the server (Netdata).
Linux updates the load average every 5 seconds. In fact, it actually updates every 5 seconds plus one "tick"
sched/loadavg.h:
#define LOAD_FREQ (5*HZ+1) /* 5 sec intervals */sched/loadavg.c
* The global load average is an exponentially decaying average of nr_running + * nr_uninterruptible. * * Once every LOAD_FREQ: * * nr_active = 0; * for_each_possible_cpu(cpu) * nr_active += cpu_of(cpu)->nr_running + cpu_of(cpu)->nr_uninterruptible; * * avenrun
% grep "CONFIG_HZ=" /boot/config-$(uname -r)CONFIG_HZ=250This means that every 5.004 seconds (5 + 1/250), Linux calculates the load average. It checks how many processes are actively running plus how many processes are in uninterruptable wait (eg. Waiting for disk IO) states, và uses that to lớn compute the load average, smoothing it exponentially over time.
Say you have a process that starts a bunch of subprocesses every second. For example, Netdata collecting data from some apps. Normally, the process will be very fast và won"t overlap with the load average check, so everything is fine. However, every 1251 seconds (5.004 * 250), the load average update interval will be an exact multiple of one second (that is, 1251 is the least common multiple of 5.004 & 1). 1251 seconds is 20.85 minutes, which is exactly the interval I was seeing the load average increase. My educated guess here is that every 20.85 minutes, Linux is checking the load average at the exact time that several processes are being started và are in the queue khổng lồ run.
I confirmed this by disabling netdata và manually watching the load average:
while true; vày uptime; sleep 5; done
After 1.5 hours, I did not see any similar spikes. The spikes only occur when Netdata is running.
So... In the end... The phầm mềm that I was using for monitoring the load was the one responsible for causing it. Ironic. He could save others from death, but not himself.
It turns out other people have hit similar issues in the past, albeit with different intervals. The following posts were extremely helpful:
Reported it lớn the Netdata devs here: https://github.com/netdata/netdata/issues/5234. In the end, I"m not sure if I"d call this a bug, but perhaps netdata could implement some jitter so that it doesn"t perform checks every one second exactly.
Select page...Get started › InstallationGet started › Models và Languages
Get started › Facts và Figures
Get started › spa
Cy 101Get started › New in v3.0Get started › New in v3.1Get started › New in v3.2Get started › New in v3.3Get started › New in v3.4Get started › New in v3.5Guides › Linguistic Features
Guides › Rule-based Matching
Guides › Processing Pipelines
Guides › Embeddings & Transformers
Guides › Training Models
Guides › Layers & mã sản phẩm Architectures
Guides › spa
Cy Projects
Guides › Saving và Loading
Guides › Visualizers
Resources › Project Templates
Resources › v2.x Documentation
Resources › Custom Solutions
Get started
Guides
Visualizers
Resources
Visualizers
Visualize dependencies & entities in your browser or in a notebook
Visualizing a dependency parse or named entities in a text is not only a fun NLPdemo – it can also be incredibly helpful in speeding up development anddebugging your code & training process. That’s why our popular visualizers,displa
Cy anddispla
Cy ENT are also anofficial part of the core library. If you’re running a
Jupyter notebook, displa
Cy will detect this & returnthe markup in a format ready to lớn be rendered and exported.
The quickest way lớn visualize Doc is to lớn usedisplacy.serve. This will spin up a simpleweb server & let you view the result straight from your browser. Displa
Cy caneither take a single Doc or a list of Doc objects as its first argument.This lets you construct them however you like – using any pipeline ormodifications you like. If you’re using Streamlit, checkout the xemlienminh360.net-streamlitpackage that helps you integrate spa
Cy visualizations into your apps!
Visualizing the dependency parse
The dependency visualizer, dep, shows part-of-speech tags và syntacticdependencies.
Dependency example
There’s currently a known issue with the compact mode for sentences with shortarrows và long dependency labels, that causes labels longer than the arrow towrap. So if you come across this problem, especially when using custom labels,you’ll have khổng lồ increase the distance setting in the options lớn allow longerarcs.
| compact | “Compact mode” with square arrows that takes up less space. Defaults to False. Bool |
| color | Text color. Can be provided in any CSS legal format as a string e.g.: "#00ff00", "rgb(0, 255, 0)", "hsl(120, 100%, 50%)" và "green" all correspond khổng lồ the màu sắc green (without transparency). Defaults khổng lồ "#000000". Str |
| bg | Background color. Can be provided in any CSS legal format as a string e.g.: "#00ff00", "rgb(0, 255, 0)", "hsl(120, 100%, 50%)" và "green" all correspond to lớn the color green (without transparency). Defaults to lớn "#ffffff". Str |
| font | Font name or font family for all text. Defaults lớn "Arial". Str |
For a danh sách of all available options, see thedisplacy API documentation.
Visualizing long texts
Long texts can become difficult khổng lồ read when displayed in one row, so it’s oftenbetter lớn visualize them sentence-by-sentence instead. As of v2.0.12, displacysupports rendering both Doc & Span objects, aswell as lists of Docs or Spans. Instead of passing the full Doc todisplacy.serve, you can also pass in a các mục doc.sents. This will create onevisualization for each sentence.Xem thêm: Mắt Xích Chung Là Gì
Visualizing the entity recognizer
The entity visualizer, ent, highlights named entities & their labels in atext.Named Entity example
The entity visualizer lets you customize the following options:
| ents | Entity types khổng lồ highlight (None for all types). Defaults to lớn None. Optional |
| colors | Color overrides. Entity types should be mapped to màu sắc names or values. Defaults lớn . Dict |
If you specify a danh mục of ents, only those entity types will be rendered – forexample, you can choose khổng lồ display PERSON entities. Internally, the visualizerknows nothing about available entity types và will render whichever spans andlabels it receives. This makes it especially easy lớn work with custom entitytypes. By default, displa
Cy comes with colors for all entity types used bytrained spa
Cy pipelines. If you’re using custom entity types, you canuse the colors setting to địa chỉ cửa hàng your own colors for them.
The above example uses a little trick: Since the background màu sắc values areadded as the background style attribute, you can use anyvalid background valueor shorthand – including gradients and even images!
Adding titles khổng lồ documents
Rendering several large documents on one page can easily become confusing. Toadd a headline to each visualization, you can showroom a title khổng lồ its user_data.User data is never touched or modified by spa
Cy.
This feature is especially handy if you’re using displa
Cy khổng lồ compare performanceat different stages of a process, e.g. During training. Here you could use thetitle for a brief mô tả tìm kiếm of the text example & the number of iterations.
Visualizing spans
The span visualizer, span, highlights overlapping spans in a text.
Span example
The span visualizer lets you customize the following options:
| spans_key | Which spans key to render spans from. Default is "sc". Str |
| templates | Dictionary containing the keys "span", "slice", and "start". These dictate how the overall span, a span slice, and the starting token will be rendered. Optional |
| kb_url_template | Optional template to lớn construct the KB url for the entity to liên kết to. Expects a python f-string format with single field lớn fill in Optional |
| colors | Color overrides. Entity types should be mapped to màu sắc names or values. Dict |
Because spans can be stored across different keys in doc.spans, you need tospecify which one displa
Cy should use with spans_key (sc is the default).
Using displa
Cy in Jupyter notebooks
displaCy is able to detect whether you’re working in a
Jupyter notebook, và will return markup that can berendered in a cell straight away. When you export your notebook, thevisualizations will be included as HTML.
Jupyter example
Important note
To explicitly enable or disable “Jupyter mode”, you can use the jupyterkeyword argument – e.g. To return raw HTML in a notebook, or khổng lồ force Jupyterrendering if auto-detection fails.
Internally, displa
Cy imports display và HTML from IPython.core.displayand returns a Jupyter HTML object. If you were doing it manually, it’d look likethis:
Rendering HTML
If you don’t need the website server và just want to lớn generate the markup – forexample, khổng lồ export it lớn a file or serve it in a custom way – you can usedisplacy.render. It works the same way, butreturns a string containing the markup.Examplepage=True renders the markup wrapped as a full HTML page. For minified andmore compact HTML markup, you can phối minify=True. If you’re rendering adependency parse, you can also export it as an .svg file.
What’s SVG?
Unlike other image formats, the SVG (Scalable Vector Graphics) uses XML markupthat’s easy khổng lồ manipulateusing CSSor
Java
Script.Essentially, SVG lets you design with code, which makes it a perfect fit forvisualizing dependency trees. SVGs can be embedded online in an tag,or inlined in an HTML document. They’re also pretty easy toconvert.
Important note
Since each visualization is generated as a separate SVG, exporting .svg filesonly works if you’re rendering one single doc at a time. (This makes sense –after all, each visualization should be a standalone graphic.) So instead ofrendering all Docs at once, loop over them and export them separately.
Example: Export SVG graphics of dependency parses
Example
The above code will generate the dependency visualizations as two files,This-is-an-example.svg & This-is-another-one.svg.
Rendering data manually
You can also use displa
Cy to lớn manually render data. This can be useful if youwant to visualize đầu ra from other libraries, like NLTKor
Syntax
Net.If you set manual=True on either render() or serve(), you can pass in datain displa
Cy’s format as a dictionary (instead of Doc objects). There arehelper functions for converting Doc objects khổng lồ displa
Cy’s format for use withmanual=True: displacy.parse_deps,displacy.parse_ents, anddisplacy.parse_spans.
