Xen domu IO freezes

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP


Xen domu IO freezes



I installed xen in the Centos 7.4.1708 following the Xen4CentOS guide.



And then I installed 3 dom-U guest (2 CentOS7, and 1 Windows server) with full virtualization. After the initial testing when I made system production available, only the linux systems periodically hangs but Windows server system is running alright.



The xen kernel is 4.9.63-29.el7.x86_64. The dom-U linux hosts are CentOS 7 (3.10.0-693.5.2.el7.x86_64), and the windows host is Windows Server 2012 R2.
The linux kernel for dom-U hosts hangs with the following kernel hang message:


> [ 3746.780097] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 3746.780223] INFO: task jbd2/xvdb6-8:8173 blocked for more than 120 seconds.



The tasks in above message are different depending what was running at that moment.



The logs end at some point until the new reboot. Sometimes it's still possible to log on to the system, but nothing really works. It is like all IO to the virtual block devices is suspended indefinitely. Until this happens, the systems seems to work without issues.



Something like 'ls' on a directory listed before still gets a result,
but everything 'new', i.e. 'vim somefile' will cause the shell to stall. sar -u reveals hi I/O wait.



Similar problem is reported for xen for other kernel (debian/suse)[https://www.novell.com/support/kb/doc.php?id=7018590] and following their suggestion I have raised gnttab_max_frames=xxx to 256. It was stable 1 weak and then one of the dom-U hangs.



Following is the output from xl info:


release : 4.9.63-29.el7.x86_64
version : #1 SMP Mon Nov 20 14:39:22 UTC 2017
machine : x86_64
nr_cpus : 32
max_cpu_id : 191
nr_nodes : 2
cores_per_socket : 8
threads_per_core : 2
cpu_mhz : 2100
hw_caps : bfebfbff:2c100800:00000000:00007f00:77fefbff:00000000:00000121:021cbfbb
virt_caps : hvm hvm_directio
total_memory : 130978
free_memory : 68109
sharing_freed_memory : 0
sharing_used_memory : 0
outstanding_claims : 0
free_cpus : 0
xen_major : 4
xen_minor : 6
xen_extra : .6-6.el7
xen_version : 4.6.6-6.el7
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : Fri Nov 17 18:32:23 2017 +0000 git:a559dc3-dirty
xen_commandline : placeholder dom0_mem=2048M,max:2048M cpuinfo com1=115200,8n1 console=com1,tty loglvl=all guest_loglvl=all gnttab_max_frames=256
cc_compiler : gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16)
cc_compile_by : mockbuild
cc_compile_domain : centos.org
cc_compile_date : Mon Nov 20 12:28:41 UTC 2017
xend_config_format : 4



This is in my production system and beginning to impact. Any clue or debugging steps are very much appreciated.




1 Answer
1



did you install xen pv drivers from here: https://www.xenproject.org/developers/teams/windows-pv-drivers.html ?



8.2.0 are signed by Linux Foundation and install just like that on windows.






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

Arduino Mega cannot recieve any sketches, stk500_recv() programmer is not responding

Image
Clash Royale CLAN TAG #URR8PPP Arduino Mega cannot recieve any sketches, stk500_recv() programmer is not responding Yesterday we were trying to update the firmware of our 3D-printer a Tevo Tarantula i3 The printer was running Marlin 3D Printer Firmware. We installed a new module a TL-Touch Sensor, a chinese copy of the Bl-Touch an auto bedleveling touch sensor for a 3D printer. After re-configurating a new version of the firmware for our setup in the Arduino IDE we went on with uploading the sketch to the Arduino Mega that was controling our 3D printer. Before we started uploading we made sure the right COM-port was selected in the Arduino IDE and that also the right type of Arduino was selected. The Arduino Mega 2260. Then we started with compiling the firmware and then uploading it to the Mega, Somewhere in between that moment the IDE freezed and crashed. As a response we removed the USB cable that was connected from the PC uploading the firmware onto the Arduino Mega, (This was a st...
Read more

Visual Studio Code: How to configure includePath for better IntelliSense results

Image
Clash Royale CLAN TAG #URR8PPP Visual Studio Code: How to configure includePath for better IntelliSense results so I am a complete beginner to using Visual Studio Code and I have no clue what I am doing. I've searched around (maybe not enough), but I cant find just a simple explanation for someone like me on how to configure the c_cpp_properties.json file that I am redirected to whenever I click on the yellow lightbulb next to a line that is underlined with a green squiggle. Lightbulb example c_cpp_properties.json I just want to know what to put in the .json to make IntelliSense work properly This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. 2 Answers 2 From: https://code.visualstudio.com/docs/languages/cpp Below you can see that the MinGW C++ include path has been added to browse.path for Windows: { "name": "Win32...
Read more

C++ virtual function: Base class function is called instead of derived

Image
Clash Royale CLAN TAG #URR8PPP C++ virtual function: Base class function is called instead of derived I have following code snippet: using namespace std; class base { public: virtual void print(char a){ std::cout<<" Base "<<std::endl;} }; class derived :public base { public: void print(floata) {std::cout<<" Derived "<<std::endl;} }; int main() { base* d = new derived; d->print(1.5); return 0; } Output is "base" Why output is coming from base function and not from derived one By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Read more